Defining a holographic object allowance area and movement path

ABSTRACT

A method, system and computer readable program storage device for controlling placement of a holographic object in a specified area. In an embodiment, the method comprises identifying, by one or more processors, at least one person in a specified area; determining by the one or more processors, a sight zone in the specified area for the identified at least one person; creating, by a holographic projector, a holographic object in the specified area; and controlling, by the one or more processors, a location of the holographic object to keep the holographic object out of said determined sight zone. In an embodiment, a group of persons are identified in the specified area as having no interest or a low interest in the holographic object, and an aggregate sight zone is determined for this group of persons. An allow zone, outside this aggregate sight zone, is determined for the holographic object, and the controlling a location of the holographic object includes keeping the holographic object in the allow zone.

BACKGROUND

This invention, generally, relates to creating holographic objects, andmore particularly, to controlling the placement and movement of theholographic objects. Even more specifically, embodiments of theinvention relate to controlling the placement and movement ofholographic objects, generated by or for one or more persons in an area,to avoid creating obstacles to the views of other persons in the area.

A holographic object is a three-dimensional object that can be createdin midair. Using multiple holographic projectors installed in a room orenvironment, participating users can share and manipulate informationand play games with holographic objects. People in the surrounding areaof the holographic image can view the object. In some cases, the peoplein the surrounding area may want to view the object or watch othersinteract with the three-dimensional object.

SUMMARY OF THE INVENTION

Embodiments of the invention provide a method, system and computerreadable program storage device. In an embodiment, the method comprisesidentifying, by one or more processors, at least one person in aspecified area; determining by the one or more processors, a sight zonein the specified area for the identified at least one person; creating,by a holographic projector, a holographic object in the specified area;and controlling, by the one or more processors, a location of theholographic object to keep the holographic object out of said determinedsight zone.

In an embodiment, the identifying at least one person in the specifiedarea includes identifying a group of persons in the specified area asnot being interested in, or having a low interest in, the holographicobject; and the determining a sight zone in the specified area includesdetermining an aggregate sight zone for said group of persons. In thisembodiment, the controlling a location of the holographic objectincludes determining an allow zone for the holographic object outside ofthe aggregate sight zone, and keeping the holographic object in theallow zone.

In an embodiment, a plurality of persons are in the specified area; andthe identifying at least one person in the specified area includesseparating the plurality of persons in the specified area into a firstgroup and a second group based on defined attentiveness of the pluralityof persons to the holographic object, said first group of persons havingan attentiveness to the holographic object below a specified level. Inthis embodiment, the determining a sight zone in the specified areaincludes determining an aggregated field of view for said first group ofpersons, and determining the sight zone from said aggregated field ofview.

Embodiments of the invention determine an allowed zone in which tocreate a projected three-dimensional holographic object, in mid air, ofdifferent sizes and shapes within the allowed zone, and that does notcreate an obstacle to other users in the environment who are engaged indifferent activities requiring a line of sight.

Embodiments of the invention provide the ability to define a path inallowed zones in which a three-dimensional holographic object can bemoved and manipulated with recalculation of different sizes and shapeswithin the allowed zones. When the holographic object is kept in thispath, the holographic object does not create an obstacle to other usersin the environment who are engaged in different activities requiring aline of sight.

Embodiments of the invention provide a method and system by which:

-   -   Cameras are used in a holographic environment to identify an        active user focus, attentiveness, viewing angles, and obstacles;    -   Rendering of holographic objects over other obstacles is        avoided;    -   Zones are defined in which to create projected three-dimensional        holographic object movement without obstructing active user        views in the environment; and    -   A path is defined through the zone to allow projected        three-dimensional holographic object movement without        obstructing active user views in the environment.

Embodiments of the invention avoid rendering of holographic objects overother obstacles. For example, having an object generated half waythrough a coffee table would look odd, and embodiments of the inventionavoid other obstacles using a camera setup.

Existing holographic systems are built independent of each other.Embodiments of the invention address the problem of users engaged inmultiple activities of different types interacting in a common areaincluding three-dimensional holograms, to determine users focus andattention. In embodiments of the invention, a holographic object iscreated in an area that does not obstruct the viewing angle of otherusers within the environment, and the holographic object does notobstruct the views of these other users when the holographic object isbeing moved.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 depicts an example of how a holographic object created by oneuser can be an obstacle to another user.

FIG. 2 illustrates a holographic object creation system in accordancewith an embodiment of the invention.

FIG. 3 shows a method in accordance with an embodiment of the invention.

FIG. 4 depicts a user's primary focus area.

FIG. 5 shows an example of an aggregated primary field of view ofmultiple users.

FIG. 6 depicts how, in an embodiment of the invention, the shape,orientation, and dimension of holographic objects can change whilemoving.

FIG. 7 illustrates a network that may be used in the system of FIG. 2.

FIG. 8 schematically shows a device that may be used in the network ofFIG. 7.

FIG. 9 shows a processing unit that may be used in the system of FIG. 2and the method of FIG. 3.

DETAILED DESCRIPTION

By using multiple holographic projectors installed in a room orenvironment, participating users can share and manipulate informationand play games with holographic objects. People in the surrounding areaof the holographic image can view the object. In some cases, the peoplein the surrounding area may want to view the object or watch othersinteract with the three-dimensional object.

In other cases, however, some people in the surrounding area may beinterested in performing or engaging in a different activity within thesame environment, such as engaging in a discussion with others in aroom, watching a movie, browsing through Internet content, or looking ata television. The holographic object created in the area may interferewith the activities these people want to engage in.

Embodiments of the invention address problems with three-dimensionalholographic image projection in multi person and activity environments.In such environments, a created three-dimensional holographic object maybe an obstacles to other users' views. In environments with other usersengaged in activities in the same area, a three-dimensional holographicimage created in mid air could be an obstacle to another user or usersengaged in a different activity in the same room. In addition, themovement of a three-dimensional holographic object may become anobstacle to other users' views. During the viewing and manipulation ofthree-dimensional holographic objects, the objects can move in mid air,and thus the paths these holographic objects take could obstruct theviews of others engaged in other activities.

FIG. 1 depicts an example of how a holographic object created by oneuser can create an obstacle to another user. Users A 102 and B 104 areplaying with holographic objects 106 in a game. As per the logic in thegame, the holographic objects are moving in the surrounding area. Inthis example, users C 110, D 112 and E 114 are watching TV 116. Theholographic object is an obstacle in the primary field of view of theuser C 110 watching the television. In this game, a holographic objectcan move its position in mid air, and thus the holographic object, intime, could potentially also impact the views of users C, D and E.

There is a need for a method and system by which the holographic objectsare created in mid air in a way that does not create an obstacle, andmove in way that does not create an obstacle, to other users engaged inother activities within the environment.

In embodiments of the invention, a new dynamic holographic image isgenerated in real time, that is then displayed in a manner that, whenviewed by users, is not impeded by any physical object in the line ofsight by each respective user. This modifies/improves the userexperience with a display/display surface. This, in embodiments of theinvention, is achieved by calculating particular zones, determining eyeplacement of each respective user and objects within the line of sightof the gaze of each respective user. The dynamic holographic image isthen projected and modified accordingly in response to each user's lineof sight/gaze/eye placement.

FIG. 2 illustrates a system 200 in accordance with an embodiment of theinvention. In this embodiment, system 200 comprises holographicprojection device 202 (i.e., a smart television, game console, tablet,etc.), network communications 204 with controlling or peer holographicdevices 206, projection of 3D Holographic objects, and one or morecameras 210 that is used to detect viewing angles of participants. Inthis system, communication between devices is used to inform the devicesof projection coordinates and movement paths for holograms.

System 200 also includes a processing unit 212. In embodiments of theinvention, processing unit 212 is provided with a group of softwarecomponents 214, 216 and 220. Component 214 is used to identify an activeuser focus, attentiveness and viewing angles, and component 216 is usedto define zones in which to create projected three-dimensionalholographic objects without obstructing the active user view in theenvironment. Software component 220 is used to define a path through thedefined zones to allow projected three-dimensional holographic objectmovement without obstructing the active user view in the environment. Anexternal camera or cameras can be added to the environment to augmentmulti activity/user environment viewing angles.

FIG. 3 shows a method in accordance with an embodiment of the invention.This embodiment of the method comprises four major steps:

-   -   Step 0: Prerequisites and pre-configuration;    -   Step 1: Identify active user focus, attentiveness and viewing        angle;    -   Step 2: Define zones in which to create a projected        three-dimensional holographic object without obstructing active        user view in the environment; and    -   Step 3: Define a path through the zones to allow projected        three-dimensional holographic object movement without        obstructing active user view in environment.

Each of these steps includes a plurality of component steps orsub-steps.

Step 0 includes two sub-steps: Prerequisites 302 and Pre-configured 304.At the Prerequisites step 302, the components of the system 200 areassembled, as for example shown in FIG. 2. At the Pre-configuration step304, the components of the system are configured for the desiredoperation, and this step, in turn, includes Peer-to-Peer Communicationwith devices 306, and Attentiveness Threshold 310. Components of system200 establish Peer-to-Peer Communication with devices in sub-step 306,and this Peer-to-Peer communication includes configuration of thedevices of the system in a peer-to-peer network (such as wifi orBluetooth) to provide projection coordinates of a hologram.

At the Attentiveness Threshold sub-step 310, one or more thresholds usedto determine the attentiveness of persons are determined. For instance,the system 200 can be configured to determine a number of seconds aperson can look away from a specified viewpoint as a measure ofattentiveness. The system can also be configured to determine a numberof times per minute or other defined period (i.e., 15 min., 30 min., 1hr., etc.) that a user can look away as a measure of attentiveness.These thresholds can be configured per user or for all users.

Step 1, as mentioned above, is to identify user focus, attentiveness andviewing angles. This step includes a number of sub-steps, and a firstsub-step 320 is the capture of viewing data from cameras 210. In thissub-step 320, cameras 210 installed in the holographic projectionecosystem identify the focus direction of various users. This can changeover time in an environment with more than one object/person of focus(i.e., television, portable holographic gaming system, interaction withother people in room. Cameras 210 may also be used to determine theusers' attention levels to the focus area based on thresholds configuredfor attentiveness, and cameras 210 may be used to observe other objectsin the environment that would obstruct the view of the projectedhologram. The data from the cameras 210 is streamed to the hologramprojection device or devices 202.

Holographic content can be classified as confidential or nonconfidential, or rated for consumption in a way similar to the way thatmovies are rated for content (ie. G, PG, PG-13 R). In embodiments of theinvention, users within a given area are identified from facial, voiceand other forms of identification (i.e., login to game console) and theuser's security profile will determine what type of content a user canview, either based on access to classified content or rating of contentfor a given age. Content displayed in the holographic object isevaluated based on the contents classification and content rating. Incases in which the content displayed is deemed appropriate for userswithin the viewing angle, the content is displayed. In cases in whichthe content is deemed not appropriate, the content and/or the zone withusers not authorized to see the content, is blocked from projectinghologram into that area. In cases in which the content is deemedsensitive, the primary user or owner of the content is provided amechanism to over-ride or provide consent for other users to see thecontent (i.e., personal email, electronic messaging service, etc.).

In embodiments of the invention, there are video cameras and microphoneswithin the environment capturing input of the users and interactionswith various devices and each other. The video content and audio contentare streamed to a cognitive/AI system to process to determine theviewing angles and interactions among the users. Using known procedures,a video can be used to determine the viewing angle of a user andtherefore determine what object they are looking at within anenvironment. In case of audio, the cognitive/AI system can use naturallanguage processing to determine if users are engaged in a conversationwith each other in addition to the viewing angle they are each lookingat each other. Procedures for capturing audio and video content and fordetermining viewing angles are known in the art.

At the sub-step 322, software installed in one or more devices in theholographic ecosystem analyzes the gathered video and image data to findthe primary field of view of each and every user based on their facialand focus direction. The holographic ecosystem includes the inventioncomponents in the system 200 and the environment such as externalcameras. The software on processing unit 212 receives capturedvideo/images from cameras in the environment of the users' lines ofsight or fields of view in the environment for analysis.

In the analysis, the software is receiving video/image input fromcameras in the environment, and location awareness. The softwaredetermines the location of the user(s) within the room. This can be donefrom location awareness software in system 200 or from cameradetermination of the distance from a location of users/devices. Softwarecan stream video/images of objects to be identified using objectrecognition techniques locally or streamed to a remote cognitive systemfor image analytics. Software determines what the user is looking atfrom eye gaze detection. Any suitable procedure may be used to determinewhat a user is looking at from eye gaze detection, and a number ofsuitable procedures are known in the art; Also, software can determinehow long a user is looking at an object based on eye gaze.

FIG. 4 depicts a user's primary focus area. A user's primary focus arealies on the near-peripheral view. Thus, if a user has focused theirattention on watching specific content, then the content should be inthe near-peripheral vision area. A user can also view the contentoutside of the primary focus area. A number of procedures are known fordetermining the primary focus area of a user, and any suitable proceduremay be used in embodiments of the invention.

With reference to FIG. 3, at sub-step 324, software installed in theholographic projection ecosystem identifies the users who are notinteracting with the holographic objects, and accordingly will calculatethe aggregated primary field of view of those users. As discussed above,embodiments of the invention use eye gaze to determine what a user islooking at, the user's line of sight, and how long (attentiveness) aperson is looking at an object. A user can be identified as notinteracting with the holographic object when the user does not have eyegaze focus on the object, or does not have the object in his or her lineof sight, or is not directly interacting with the holographic object.These will be known when the user has no interactive gestures orcontrols with the object.

FIG. 5 depicts an aggregated primary field of view of multiple userslooking at a common object. This aggregated primary field of view iscomprised of three separate primary fields of view 502, 504 and 506 ofeach of users C. D and E respectively. There are situations whendifferent people are looking at different areas.

In the example of FIG. 5, three users (C, D, E) are positioned indifferent locations within a room and are watching content on thedisplay 510. The aggregated primary field of view is calculatedindividually to the object of focus, and then mapped collectively in thespace such that holographic objects are not projected within the line ofsight of the users observing content on the display 510.

The following is an example of an aggregated primary field of viewcalculation. The location of the object of focus within the room isdetermined. That object or device, such as a smartTV, can have locationawareness information accessible to software used in embodiments of theinvention. Alternatively, the location of the object can be determinedfrom eye focus and a distance calculation. User C's primary field ofview is calculated based on C′s current position and his or her line ofsight to the object. This is repeated from the location of each user(i.e., D, E) to the location of the object of the user's eye focus. Theprimary field of view for each user is plotted to create an aggregatedprimary field of view relative to the object of each user's focus. Anysuitable procedure may be used for a field of view calculation, and anumber of suitable procedures are known in the art.

Plotting a holographic object in the aggregated primary field of viewwill create a disturbance to one or more of these users. If any userwants to create the holographic object, then, in embodiments of theinvention, the projection ecosystem plots the holographic objectsoutside of the aggregated primary field of view of all the participatingusers. When the holographic objects are moving, the projection system200 attempts to automatically find a route for the holographic objectssuch that no users are disturbed or obstructed by the moving holographicobjects.

When determining possible areas to project the 3D object within thespace, there is a need to consider the users' primary field of view suchthat they can view the object as well as a need to consider if anotheruser can view the object if multiple person interaction is required.Depending on the task and content type, the object may require a primaryfield of view (i.e., central 5 degrees, or paracentral 8 degrees), or iflesser focus is required near, peripheral to the secondary user may beacceptable (i.e., 30 degrees).

With reference to FIG. 3, step 2 defines zones in which to create aprojected three-dimensional holographic object without obstructingactive user view in the environment. At sub-step 326, the holographicecosystem aggregates the gathered primary field of view of each of theusers, and accordingly the restricted holographic object creation zoneis created. At 330, zone definition and dimensions considers avoidingrendering of holographic objects over other obstacles. For example,having a holographic object generated half way through a coffee tablewould just look odd and would probably not be desirable, and so theprojection system 200 also looks to avoid other obstacles using thecamera setup. The defined zone dimensions created for the hologram maybe used as input, at sub-step 332, to determine the size and shape ofthe created hologram so as to not obstruct the views of users in theenvironment.

Step 3 defines a path through the zone to allow projectedthree-dimensional holographic object movement without obstructing activeuser view in the environment. Each zone is a representation of areasthat may be defined as a primary field of view for users and objects, asdefined by a plotted aggregated primary field of view. Conversely, azone can be defined as space that does not have primary field of viewand thus can have holographic objects projected in the space withoutcreating an obstruction. With reference to FIG. 5, the aggregatedprimary field of view shows areas in which the holographic object shouldnot be projected for the scenario in which users are watching atelevision display. When adding another scenario, such as hand heldgaming device, in same space, it is desirable to prevent those devicesfrom projecting a holographic object that would obstruct the televisiondisplay. The space or zone plotted outside the aggregated primary fieldof view is a zone in which a holographic object from a hand held gamedevice could be projected. The eye gazes of users are monitored fromcameras in the environment. In embodiments of the invention, this is aiterative process. If the users get up and change positions or peopleenter or exit the space, the calculations may be redone.

At sub-step 334, the holographic projection defines the aggregatedprimary field of view of those users as a restricted zone to holographicobject movement. Accordingly, the holographic projection ecosystemidentifies the remaining areas or zones where the holographic objectscan travel.

At sub-step 336, components 214, 216, and 220 communicate withprocessing unit 212 to automatically change the shapes or orientationsof the holographic objects while moving the holographic objects aroundthe environment based on the dimension of the zone and the shapes ororientations of the holographic objects. Some allowed paths may benarrow or uneven spaces, and thus to ensure the smooth movement of theholographic objects, the holographic projectors recalculate the shapes,dimensions, or orientations of the holographic objects, and theholographic objects travel accordingly. At sub-step 340, the holographicprojection system identifies the route of the moving holographic objectsin such a way that no users are impacted, and the projection system alsodecides the dimensions, shapes, and orientations of the holographicobjects.

FIG. 6 depicts how the shapes, orientations, and dimensions ofholographic objects 602 may change while travelling from one point 604to another point 606 without disturbing other persons in the area.

When the 3D holographic object exists in a zone, the holographic imagehas specific dimensions and shape that define the space that it takes upwithin the current zone. When an obstruction is determined and the newzones are determined to be free of obstruction, there is a need totransition the 3D object from the existing zone to the new zone. Thereare a number of options for this. As a first example, the holographicobject is moved from one zone to another with brief obstruction whilebeing moved. As a second example, the holographic object takes an objectmovement path to the new zone without obstruction. In this case, thepath between the original zone and new zone is evaluated to determine ifthe size of the path is equal or greater than the current size of the 3Dholographic object. If the size of the path is not equal to or greaterthan the current size of the 3D holographic object, the holographicobject is resized, for example, to the maximum allowable size throughthe movement path and then the holographic object is resized to itsoriginal size once in the new zone such that the holographic object doesnot create an obstruction during movement.

As mentioned above, system 200 of FIG. 2 comprises networkcommunications with controlling or peer holographic devices, and FIG. 7illustrates a network communication 700 that may be used in embodimentsof the invention. As represented in FIG. 7, a variety of devices may beused to obtain data to determine the viewpoints and othercharacteristics of the persons in an area. These devices include, forexample, camera 210. Other devices may also be used including, forexample, a cell phone 702 and sensor or detectors, represented at 704and 706.

The devices of network 700 may communicate directly or indirectly witheach other and with holographic projector 202 of system 200; and thedevices of the network may be provided with suitable input and outputcomponents to communicate with each other. A communication system 710may be provided to facilitate communications between the devices of thenetwork. Network 700 may also be provided with a server (not shown) toreceive, process and transmit data from and to other devices in thenetwork.

If desired, network 700 may be connected to the Internet to receive datafrom and to transmit data to the Internet, and the network 700 may beprovided with additional or alternative devices than are shown in FIG. 7to facilitate collection of data and the transmission of data among andbetween the devices of the network. As will be appreciated by those ofordinary skill in the art, the network 700 of FIG. 7 is intended as anexample, and not as a limitation for the invention.

FIG. 8 illustrates an example of a device 800 that may be used inembodiments of the invention in network 700 to collect and/or processdata about persons in an area. While internal components of devices 800can be embodied with different hardware configurations, a basichigh-level configuration for internal hardware components is shown asplatform 810 in FIG. 8. The platform 810 can receive and executesoftware applications, data and/or commands transmitted over a networkinterface and/or a wired interface. The platform 810 can alsoindependently execute locally stored applications. The platform 810 caninclude one or more transceivers 812 configured for wired and/orwireless communication (e.g., a Wi-Fi transceiver, a Bluetoothtransceiver, a cellular transceiver, a satellite transceiver, a GPS orSPS receiver, etc.) operably coupled to one or more processors 814, suchas a microcontroller, microprocessor, application specific integratedcircuit, digital signal processor (DSP), programmable logic circuit, orother data processing device, which will be generally referred to as theprocessor 814.

The processor 814 can execute application programming instructionswithin a memory 816 of the device 800. The memory can include one ormore of read-only memory (ROM), random-access memory (RAM), electricallyerasable programmable ROM (EEPROM), flash cards, or any memory common tocomputer platforms. One or more input/output (I/O) interfaces 820 can beconfigured to allow the processor 814 to communicate with and controlvarious I/O devices and any other devices, such as sensors, actuators,relays, valves, switches, etc. associated with the device 800. While notshown explicitly as part of the device 800, the device may include oneor more external antennas and/or one or more integrated antennas thatare built into an external casing or housing, including but not limitedto Wi-Fi antennas, cellular antennas, satellite position system (SPS)antennas (e.g., global positioning system (GPS) antennas), and so on.

With reference to FIG. 9, a block diagram of a data processing system900 is shown. Data processing system 900 is an example of a processingunit that may be used in, or with system 200 of FIG. 2. Data processingsystem 900 may also be used in or with network 700 of FIG. 7 or in ordirectly with device 800 of FIG. 8.

In this illustrative example, data processing system 900 includescommunications fabric 902, which provides communications betweenprocessor unit 904, memory 906, persistent storage 908, communicationsunit 910, input/output (I/O) unit 912, and display 914.

Processor unit 904 serves to execute instructions for software that maybe loaded into memory 906. Processor unit 904 may be a set of one ormore processors or may be a multi-processor core, depending on theparticular implementation. Memory 906 and persistent storage 908 areexamples of storage devices. Memory 906, in these examples, may be arandom access memory or any other suitable volatile or non-volatilestorage device. Persistent storage 908 may take various forms dependingon the particular implementation. For example, persistent storage 908may be a hard drive, a flash memory, a rewritable optical disk, arewritable magnetic tape, or some combination of the above.

Communications unit 910, in these examples, provides for communicationswith other data processing systems or devices. In these examples,communications unit 910 is a network interface card. Communications unit910 may provide communications through the use of either or bothphysical and wireless communications links. Input/output unit 912 allowsfor input and output of data with other devices that may be connected todata processing system 900. For example, input/output unit 912 mayprovide a connection for user input through a keyboard and mouse. Theinput/output unit may also provide access to external program code 916stored on a computer readable media 920. In addition, input/output unit912 may send output to a printer. Display 914 provides a mechanism todisplay information to a user.

Those of ordinary skill in the art will appreciate that the hardware inFIG. 9 may vary depending on the implementation. Other internal hardwareor peripheral devices, such as flash memory, equivalent non-volatilememory, or optical disk drives and the like, may be used in addition toor in place of the hardware depicted in FIG. 9.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The description of the invention has been presented for purposes ofillustration and description, and is not intended to be exhaustive or tolimit the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope of the invention. The embodiments werechosen and described in order to explain the principles and applicationsof the invention, and to enable others of ordinary skill in the art tounderstand the invention. The invention may be implemented in variousembodiments with various modifications as are suited to a particularcontemplated use.

The invention claimed is:
 1. A method comprising: creating, by aholographic projector, a three-dimensional (3D) holographic object inmid-air in a specified area of the real world environment; identifying,by one or more processors, a multitude of persons engaged in activitiesrequiring line-of-sight viewing of objects within the specified area ofthe real world environment, the identifying including identifying afirst group of persons from said multitude of persons in the specifiedarea as having an aggregated line of sight viewing of another object anda collective attentiveness to the 3D holographic object below aspecified level indicating no interest or a defined low interest inline-of-sight viewing of the 3D holographic object and includingidentifying a second group of persons engaged in an activity involvingsaid 3D holographic object; determining by the one or more processors,an aggregate sight zone corresponding to aggregated said line-of-sightviewing of the another object in the specified area for the identifiedfirst group of persons while engaged in said activities; andcontrolling, by the one or more processors, the holographic projector tocontrol a location of the 3D holographic object to keep the holographicobject outside of said determined aggregate sight zone, said controllingthe holographic projector comprising: determining an allow zone for the3D holographic object outside of the aggregate sight zone; keeping theholographic object in the allow zone; and moving the holographic objectfrom one location in the allow zone to another location in the allowzone, including identifying a path within the allow zone, between saidone location and said another location, wherein: the holographicprojector controlling a location of the holographic object includeskeeping the 3D holographic object in said path as the 3D holographicobject moves from said one location to said another location while saidsecond group of persons engage in said activity involving said 3Dholographic object.
 2. The method according to claim 1, wherein thekeeping the holographic object in said path includes: changing at leastone of a size of the holographic object, a shape of the holographicobject, and an orientation of the holographic object as the holographicobject moves in said path from said one location to said anotherlocation to keep the holographic object in said path as the holographicobject moves from said one location to said another location.
 3. Themethod according to claim 1, wherein: a multitude of persons are in thespecified area; and the identifying at least one person in the specifiedarea includes identifying a plurality of persons of the multitude ofpersons in the specified area; the determining a sight zone in thespecified area includes determining lines of sight for the identifiedplurality of persons in the specified area, and using said determinedlines of sight to determine the sight zone.
 4. The method according toclaim 1, wherein: a plurality of persons are in the specified area; andthe identifying at least one person in the specified area includesidentifying at least one of the plurality of persons in the specifiedarea as having a defined interest in the holographic object below aspecified threshold.
 5. The method according to claim 1, wherein: aplurality of persons are in the specified area; the identifying at leastone person in the specified area includes separating the plurality ofpersons in the specified area into a first group and a second groupbased on defined attentiveness of the plurality of persons to theholographic object, said first group of persons having an attentivenessto the holographic object below a specified level; and the determining asight zone in the specified area includes determining an aggregatedfield of view for said first group of persons, and determining the sightzone from said aggregated field of view.
 6. The method according toclaim 5, wherein the separating the plurality of persons in thespecified area into a first group and a second group includes:generating video data of the plurality of persons in the specified area,and using the video data to determine said defined attentiveness and toseparate the plurality of persons into the first group and the secondgroup.
 7. The method according to claim 5, wherein the separating theplurality of persons in the specified area includes: identifying firstand second activities of the plurality of persons in the specified area;identifying one or more of the plurality of persons in the specifiedarea as engaged in the first activity, and identifying one or more ofthe plurality of persons in the specified area as engaged in the secondactivity; and grouping the identified one or more persons engaged in thefirst activity into the first group, and grouping the identified one ormore persons engaged in the second activity into the second group. 8.The method according to claim 5, wherein a specified level ofattentiveness comprises a threshold time duration a person looks awayfrom a specified viewpoint, said method further comprising: monitoringhow long a user looks away from a specified viewpoint to determine thatuser's attentiveness to the holographic object.
 9. The method accordingto claim 5, wherein a specified level of attentiveness comprises athreshold number of instances a person looks away from a specifiedviewpoint in a defined time period, said method further comprising:determining a number of times instances a person looks away from aspecified viewpoint in a defined time period to determine that user'sattentiveness to the holographic object.
 10. The method according toclaim 1, wherein the controlling a location of the holographic objectincludes: determining an allow zone for the holographic object outsidethe determined sight zone; and keeping the location of the holographicobject in the allow zone.
 11. A system comprising: a holographicprojector for generating a three-dimensional (3D) holographic object inmid-air in a specified area of a real world environment; a useridentification system for identifying at least one person of a multitudeof persons engaged in activities requiring line-of-sight viewing ofobjects within the specified area of the real world environment, theidentifying including identifying a first group of persons from saidmultitude of persons in the specified area as having an aggregated lineof sight viewing of another object and a collective attentiveness to the3D holographic object below a specified level indicating no interest ora defined low interest in line-of-sight viewing of the 3D holographicobject and including identifying a second group of persons engaged in anactivity involving said 3D holographic object; and a control system fordetermining an aggregate sight zone corresponding to said aggregatedline-of-sight viewing of the another object in the specified area forthe identified first group of persons while engaged in said activities,and for controlling the holographic projector to control a location ofthe 3D holographic object in the real world environment to keep the 3Dholographic object outside of said determined aggregate sight zone, saidcontrol system further determining an allow zone for the holographicobject outside of the aggregate sight zone; controlling the holographicprojector to keep the 3D holographic object in the allow zone, theholograph projector controlling: moving the holographic object from onelocation in the allow zone to another location in the allow zone,including identifying a path within the allow zone, between said onelocation and said another location, wherein: the controlling a locationof the holographic object includes keeping the 3D holographic object insaid path as the 3D holographic object moves from said one location tosaid another location while said second group of persons engage in saidactivity involving said 3D holographic object.
 12. The system accordingto claim 11, wherein the specified area includes a plurality of persons,and wherein: the control system separates the plurality of persons inthe specified area into a first group and a second group based ondefined attentiveness of the plurality of persons to the holographicobject, said first group of persons having an attentiveness to theholographic object below a specified level; and the control systemdetermines an aggregated field of view for said first group of persons,and determines the sight zone from said aggregated field of view. 13.The system according to claim 12, wherein the control system includes: acomponent for identifying first and second activities of the pluralityof persons in the specified area; a component for identifying one ormore of the plurality of persons in the specified area as engaged in thefirst activity, identifying one or more of the plurality of persons inthe specified area as engaged in the second activity, and a componentfor grouping the identified one or more persons engaged in the firstactivity into the first group, and grouping the identified one or morepersons engaged in the second activity into the second group.
 14. Thesystem according to claim 12, wherein a specified level of attentivenesscomprises a threshold time duration a person looks away from a specifiedviewpoint, said user identification system further monitoring how long auser looks away from the specified viewpoint to determine that user'sattentiveness to the holographic object.
 15. A computer readable programstorage device comprising: a computer readable storage medium havingprogram instructions embodied therein, the program instructionsexecutable by a computer to cause the computer to perform the method of:creating, by a holographic projector, a three-dimensional (3D)holographic object in mid-air in a specified area of the real worldenvironment; identifying a multitude of persons engaged in activitiesrequiring line-of-sight viewing of objects within the specified area ofthe real world environment, the identifying including identifying afirst group of persons from said multitude of persons in the specifiedarea as having an aggregated line of sight viewing of another object anda collective attentiveness to the 3D holographic object below aspecified level indicating no interest or a defined low interest inline-of-sight viewing of the 3D holographic object and includingidentifying a second group of persons engaged in an activity involvingsaid 3D holographic object; determining an aggregate sight zonecorresponding to said aggregated line-of-sight viewing of the anotherobject in the specified area for the identified first group of personswhile engaged in said activities; and controlling the holographicprojector to control a location of the 3D holographic object to keep theholographic object outside of said determined aggregate sight zone, saidcontrolling the holographic projector comprising: determining an allowzone for the holographic object outside of the aggregate sight zone;keeping the holographic object in the allow zone; and the holographicprojector controlling a moving the holographic object from one locationin the allow zone to another location in the allow zone, includingidentifying a path within the allow zone, between said one location andsaid another location, wherein: the controlling a location of theholographic object includes keeping the 3D holographic object in saidpath as the 3D holographic object moves from said one location to saidanother location while said second group of persons engage in saidactivity involving said 3D holographic object.
 16. The computer readableprogram storage device according to claim 15, wherein the specified areaincludes a plurality of persons, and wherein: the identifying at leastone person in the specified area includes separating the plurality ofpersons in the specified area into a first group and a second groupbased on defined attentiveness of the plurality of persons to theholographic object, said first group of persons having an attentivenessto the holographic object below a specified level; and the determining asight zone in the specified area includes determining an aggregatedfield of view for said first group of persons, and determining the sightzone from said aggregated field of view.
 17. The computer readableprogram storage device according to claim 16, wherein the separating theplurality of persons in the specified area includes: identifying firstand second activities of the plurality of persons in the specified area;identifying one or more of the plurality of persons in the specifiedarea as engaged in the first activity, and identifying one or more ofthe plurality of persons in the specified area as engaged in the secondactivity; and grouping the identified one or more persons engaged in thefirst activity into the first group, and grouping the identified one ormore persons engaged in the second activity into the second group.